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Abstract 


Problem- solving procedures based on the methods of combina- 
torial programming are presented for solving a class of integer 
programming problems in which all elements are zero or one, 
All of the procedures seek first a feasible solution and then 
successively better and better feasible solutions until ultimately 
one is discovered which is shown to be optimal, By represent- 
ing the problem elements in a binary computer as bits in a word 
and employing logical "and" and "'or'' operations in the problem- 
solving process, a number of problems involving several hundred 
integer variables have been solved in a matter of seconds, 
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I, INTRODUCTION 


In this paper are considered some direct algorithms for solving 


integer programming problems of the form: 


Jen 
Minimize Z= . Cy =X, 
diet jj 
jen 
Subject to: Day | 4, =R (1) 
and x, = 0 or l JEL Sieg mM 
J 
and of the form: 
jen 
Minimize Zs ies GL, 1K, 
‘tn 
jen 
Subject to: - x, A >R (1') 
jel 1a a 
and * = 0 or l jz1,2,..., 0 


a.) , each element 


where A. is an mxl column vector (a,,, a,.,..65 
=] 1 2j mj 


of which is zero or one, and R is an mxl column vector (r), To: Rete Tn) 


in which all elements are one. All ey are assumed to be nonnegative. 
As discussed by Balinski [3] » problems which can be put in 

these forms arise in such diverse contexts, for example, as that of the 

engineer in designing switching circuits, the logician in seeking a 


1 


simplest disjunctive normal form which is equivalent to a given formula’, 


and the business man in distributing goods from a central warehouse to 


a group of outlying clients. To this diversity might be added that of the 





l Por application in the operations research field see, for instance, the 
article by Root [is] 3 





industrial engineer in balancing an assembly line [20] ,» the PERT- 
CPM project manager in selecting from among feasible alternatives 
the rest means for accomplishing particular tasks [5] » and the 
computer systems analyst in retrieving information most efficiently 
from a set of files [7] ‘ 

The methods investigated for solving these problems are those 
of combinatorial programming. By combinatorial programming, we 
mean, after Rossman and Twery [19] problem-solving procedures 
based on: (i) the use of a controlled enumerative procedure for consid- 
ering (at least implicitly) all potential solutions; and (ii) the aiferseaion 
from explicit consideration of particular potential solutions which are 
known from dominance, bounding and feasibility considerations to be 
unacceptable. We limit the meaning of combinatorial programs to 
procedures based on these two concepts which are reliable in the respect 
that iwi carried out to completion they guarantee the discovery of an 
acceptable solution (in the present problem, an optimal feasible solution) 
when one exists. Alternately, the methods to be employed might be 
termed "branch and bound" after Little et al [13] or, perhaps, 
"reliable heuristic programming". 

While in general there are several basic search strategies that 
might be employed in combinatorial programs, principal attention will 
be focussed in the present paper on one wherein search is directed first 
to the discovery of a feasible aélution, and then to successively better and 
better adiale solutions until ultimately one is discovered which is shown 
to be optimal. The importance of this feature of course lies in the 


possibility of terminating problem-solving prematurely in time-consuming 


Piece 


problems with a useable although not necessarily optimal solution. 

First, attention is focussed on problems of form (1), In the 
following section the basic algorithm is described and illustrated by 
means of a small example. While completely general, the algorithm 
has been developed to capitalize on the fact that since all elements are 
either zero or one they may be represented in a binary computer as 
bits in a word and operated on with logical "and"! and "or'' operations. 
Thus, in a problem with m constraints each column vector A, can 
be represented as Km/B F696 a0 » binary words, where B is the 
number of bits per word in the computer and <> denotes the largest 
integer contained in 8 . In Section HI computational experience with 
the algorithm is discussed as obtained on an IBM 7094, a binary computer 
with B =36,. In Section IV several prospective modifications to the 
algorithm are noted, and finally, in Section V the extension to problems 
of the form (1') is outlined, 

In relationship to other algorithms the present procedures appear 
most closely akin to the additive algorithm proposed by Balas [1] for 
solving the general zero-one integer programming problem. In the 
present procedures the underlying enumeration scheme is simpler, 
employing throughout problem-solving a fixed, prescribed preordering 
of the column vectors Ag ahigs axel ons Ae . As a consequence relatively 
little problem-solving time is expended on matters of procedural planning 
and control, e.g., on performing bookkeeping tasks to record the portions 
of the tree elaborated and evaluated so far, on selecting the next variable 
x; to consider, etc. ‘With this simplicity and the manipulative economies 


achieved through representation of vectors as binary words,it has been 


=e 


possible to solve a number of problems involving several hundred vari- 


ables in a matter of seconds, 


Il. THE BASIC ALGORITHM 

As has been mentioned, a first principle of combinatorial pro- 
gramming is that of controlled enumeration of potential solutions. To 
insure reliability of the resulting’ problem-solving procedure this implies 
that all 2" of the possibilities x= (X11 %>> ters’ x) must be considered, 
at least implicitly. In the present case the underlying enumeration 
scheme used to accomplish this is an elementary tree search method. 
If we define a potential solution X"' to be lexicographically smaller than 

1 " 1 " 1 


1 Lt sea . = : : = : 
X' when a ei ee es ee x} 


r, l<r <n, _ then the underlying scheme consists simply in enumerating 


W ! 
and x,< x, for some 


all 2" possibilities in lexicographically decreasing order, where the 
ordering of the column vectors a Ay rite rAd remains fixed through- 
out problem-solving. As suggested in Figure 1, these possibilities can 

be conveniently represented in a tree structure where the j* level of 
branches represents the values for = and where each node at the terminus 
of a branch at level j represents the subset of all potential solutions 

X= (x), Xypeees i oe x,) ‘in which xy =x) 2 ae Ne i 

the values of the x; being determined by the path connecting the node 


+, 


with the origin. With the branches emanating from each node ordered 
with *, = 0 onthe right, the enumeration of potential solutions in lexi- 
cographically decreasing order corresponds to elaborating paths in the 
tree from left to right. Equivalently, therefore, enumeration may be 
viewed as entailing the successive stage by stage specification of a value 


for XJ], X2,cee, x, until a complete combination’ (xy) XgQieees xy) is 





Figure 1, Tree representation of potential solutions as generated 
by the basic enumeration procedure. 


determined, or as entailing the successive level by level selection of 
branches in the tree, one branch per level, until a terminal node is 
reached at level n. Upon reaching a terminal node the corresponding 
potential solution is evaluated and then the tree-elaboration process 
backtracks to the first node in the path for which both branches have not 


been enumerated and resumes with the second branch, When the process 


oes 


has backtracked to the origin node and both branches emanating from 


this node have been enumerated, generation and therefore problem- 
solving is complete. 
With this procedure, then, the discovery of an optimal feasible 


nth nodes 


solution is guaranteed by elaborating the complete tree of 2 
and evaluating each of the 2" resulting potential solutions. If, however, 
it becomes known with certainty during enumeration that all paths passing 
through a specific node must ultimately result in potential solutions which 
are nonfeasible or are dominated” by a feasible solution already dis- 
covered, then the further elaboration and investigation of such paths can 
be eliminated without compromising the reliability of the procedure, 
We consider first some possibilities for reducing search on the basis of 
feasibility cownitiwanons. 

To begin it is noted that the enumeration procedure proceeds by 


successively specifying a value for Xp Xgreees Ke and so on until 


ultimately a complete combination X = (x5 Xpreces Xerces x,,) is 
specified, At any stage s-l, having specified the values X,, Xz,..+5 Xp 
there remains a problem of the same form as (I): 
jen 
Minimize ye Cs. &, 
JFs og 
jen j=s-1 
Subject to: , se, pAs. SR x Aye al2 
J ertceg j —i es Lig jj ( ) 
and x; =Oorl i 2 By Ctlevisg @ 
Denoting by R_=(r,, 12 Ed: Uo thesal t en aan ae 
enoting by Roa (ty* Fy veeee Fry e column vector R - jel * = 
which results upon the completion of stage s-1, it follows from the 
nonnegativity of the elements as. that for each k, 2 > nie vee> ae 


J 


2 Solution X will be said to dominate X if bj City < ye, x 
~ ip d= J 


for all permissible values of Xoo X seer Xe Hence in specifying 


stl’ 
x, if ry, =0 for any k such that as = 1, feasibility of (2) requires 
that x, = 0. In such a case none of the "ili -l nodes in the tree 


stemming from the branch mae =1 need be explicitly considered, 


Incorporating this feasibility consideration into the enumeration 


scheme the resulting procedure may be stated as follows: 


(i) Set * F F(R) x, = F(R Die) %, = F(R) . 
jen 
If R = i 1 x, A, save X as the best feasible solution 
= j= 5 3 = 
0. 0 i= 0 
X discovered so far, setting Z = y. 1 oy 2, - Go to (ii). 
— J=. 
(ii) If there isa j, 1< j < n such that at, =1, let s be the 


largest such j and go to (iii), Otherwise problem-solving 
is complete, 


(iii) Redefine Xy = Xi X_ = XQ Gees Hg Mei fy da 


jen 
F(R Ji ei pF F(R). ER = Leak x, A, and 


son 
hel ey %j < z° save X as the best feasible solution 
discovered so far. Go to (ii). 


where 
jzs-l 
R_=R - : x, A, 
lifR @® A =A 
and = 8 = ~s 


F(R.) 


0 otherwise . 


ai Bre 


the symbol ® denoting the logical "and" operation?, 

Another feasibility consideration arises from the fact that a 
necessary eondition for the existence of a feasible solution to (2) at 
stage s is that for all k such that t = 1 there must exist at least 


one vector Ay s<t<n_ such that ane =1. One simple but effective 


means of employing this consideration is to pre-order the vectors A. 


by sets, Si» S,, Sensi sa wherein in set Ss) are placed all vectors Ax 


for which aj, =1; in set S, are placed all vectors A), not in S| 


k 


for which a, = 1; andin general, in set a are placed all vectors A, 


notin S,,S5,..., or S for which a =1, q<m. When represented in 
1 2 q-l qk 2 P 


matrix form with column vectors ApA, & dross AL so ordered, the result- 


ing structure is as shown in Figure 2 where all elements in the unshaded 


portion of the matrix are zero, 


1 2 m 
Aeon |A wert. Pass lace 
Arey, Arie, Pia al 








Figure 2. Structural form of matrix of vectors Aye Bos seo,A after 
preordering by sets, All elements in the ™ unshaded 
portion of the matrix are zero, 





greeee bd) 


9 JE 1 2p. gs 


As applied to vectors A= (a); Aareees a) and B = (b,, b 
i, if aj=bj=l 
0 otherwise 


A similar pre-ordering of vectors has been employed by Lawler [12] 
in an algorithm for solving the set covering problem (see [3] , p. 286). 


we mean by A ® B=C that e5=f 


Denoting by Jy the number of vectors in S, there exists with this 


=d 
ordering no vector Aw KS Tye lass j for which Agials There- 


’ 
fore at any stage s there exists no feasible solution to the remaining 
problem (2) if a =lfor any i such that J, <s: in such cases the 
search process can "backtrack" immediately without considering further 
the variables x,, Xg4jse0+s Xy > 

With this same ordering of vectors a variant of the first feasibility 
consideration which has considerable "look ahead" power can readily be 
employed. Suppose stage s has just been completed wherein xg and 


Ro have been determined, If we denote by t the smallest index i 
stl 


such that r; °° =1 then explicit investigation can immediately jump to 


stage s' = Jey +1, and resume with Ry = R, since by the ordering 


+1 
of A, we must necessarily have X_ =0 for s<k< wa . 
In addition to possibilities for reducing search which stem from 
feasibility considerations there are possibilities based on dominance 
considerations, If the cost of the best feasible solution X ° discovered 
so far is z° then in subsequent search no explicit consideration need be 
given potential solutions for which it is known that Z > z . At stage 
s, then, investigation may be restricted to potential solutions X for 
which 


jen jes-1l 
0 — 
-, c. & <2 = x [ole ae 3 
Ujes joj Yjet ded Me 
In essence (3) may be viewed as simply another constraint defining a 
"feasible" solution and the solving of the optimization problem perceived 


as being accomplished through the solution of a succession of (augmented) 


feasibility problems. 


SuilOhe 


j=s-l 


Denoting by z_ the cost (z° - 7 c.x.) we have first, by 
s j= J Jj 
analogy with (and in addition to) the first feasibility consideration, that 
p= i 
for any particular vector A. x= 0 if c.> Zee Secondly, assuming || 


the ordering of vectors by sets as described above, we have a related 
dominance consideration having a degree of "look ahead" power as 
follows. At the completion of stage s suppose t is the smallest index a 
i such that oe =1. Then there exists no feasible solution to the 
augmented problem A(2) and (3)) defined at stage s+l1 if i > a r 
where = Jen a = Bi bie Ties (2) requires that x; =1 s some 
JS <j< J, - In such cases, the search procedure can backtrack im- 
mediately. 

The third dominance consideration is one whose implementation 
requires slightly more problem-solving time but appears to have a 
fairly high degree of "look ahead" power. Considering again the problem 
(2) at stage s suppose the constraint equations are summed to form the 
single constraint ee h, af oe ay? where 5, is the number of unit 
elements in vector ay ee ai » and m J is the number of constraints 


Ss 


=m 
in (1) remaining to be satisfied at stage> s secs r. . Using this single 
8 2 i=l i 8 8 


5 
constraint the following subproblem may be formed: 


jan 


Minimize y c = 7 

jes “575 7s , 
Subject to: lea = 4 
ubject to: = eh =m, (4) © 
and yj =Oor 1, jrs, stl,.s.,n 


* A * 
If Z, denotes the minimum value for (2) then Z5< Z. since in (2) 


In the terminology of Glover [10] this constraint is a "surrogate 
constraint". 


3 Ble 


feasibility requires that each of the m individual constraints be satisfied 
also. Therefore if at any stage s, z, > zo the search procedure can 
backtrack immediately since there can exist no solution to (2) resulting 
in a feasible solution better than ae A 

The subproblem defined in (4), being one-dimensional knapsack 
problem, can in practice be solved by any one of a number of algorithms, 
e.g., by dynamic programming [8] , by combinatorial programming 
[ 9, 14] - In the present context an approach of the latter type might be 
preferable that has the desirable characteristic that search proceeds first 
to the discovery of a feasible solution Y and then to successively better 
feasible solutions until ultimately an optimal feasible solution ay is 


discovered; with such an algorithm knapsack problem-solving can then 


be terminated at any time a feasible solution Y is discovered for which 


A 
Alternately, rather than actually solve the knapsack problem for Z, 


Aa A 
we might simply determine a lower bound 2. for 4, » and check at 
Q 


A 
each stage s whether 252 Zee While less powerful, this test requires 
the expenditure of considerably less problem-solving time to implement, 


For suppose that vectors As A apeeee An were ordered so that 


stl 
€(s)/ big) < rat)! Brad) S10 Crt Bin) . Then by weakening the 


integer requirements be =O orl in (4) to 0 < y, < 1 a lower bound on 
= %- 


the cost of satisfying u constraints at stage s is given by the function 
kj a ats oe 
kel “(ky AP° 8 (jy Coat SG) * 


If, then, at stage s, B 3) > Ze the search procedure can backtrack, 


B, (x) as shown in Figure 3, where By) = “ 


Or, an alternative requiring even less implementation time is 


one in which the quantity (6 a u) is used as a lower bound on 6 3) F 


= 12 


= min 


where 6 3 
8s j28s 


{ o;/ aj . In this case the test then becomes simply 
"6 5° ™s2 45 ?'", This alternative is investigated in the following 
section, 

Finally, there are a number of heuristics that might be employed 
for ordering the vectors A, within each set Sas, for instance, 
"increasing cost, C3 ", or "decreasing number of constraints satisfied, 


ar .'' Por present purposes we employ a heuristic representing a composite 
of these: vectors will be ordered such that ay coord in the order implies 
o/h < cy. / hy, 3 

Imbedding these various considerations and heuristics in the basic 
enumeration procedure there results an algorithm whose logic is shown 
in the flow chart of Figure 4, In this procedure record is maintained of 


only the nonzero elements in the partial solution XS = (*)> Xo rae 1X 5) 


defined at each stage s, this being accomplished simply by maintaining 





Bay) ae 


Figure 3. Lower bound B _(u) at stage s on the cost of 
satisfying u constraints. 








2 ies 


Start 





a. For all vectors BS determine ey the smallest index i, 1<i<m, such 
that a,. = 1 and sort the vectors into sets S),5,,.++5 S_ such that A 
ij 2 m =k 
is in set ee when ede 
b. Order the sets of vectors such that Sj <S, Sass Ss Sin ‘ 
c. For each vector compute @, = o,/h, and order allvectors within each 


set Be such that A ea in the ordering implies ay 5 & 


k 
d. Determine the number of vectors Jq in set Sq q=1,2;...,m and the 


t 


quantities J, = » jie GS) 12h we eM. 





i=l 
e. For each set S determine 9 = min {c;}. 
a Vo Ig-1<isd 
"qd 
f. For each vector A; determine 6. = _min a), 
=J J j<i<n J 
° 
Set Z”~ =eo, s=2, m= Cys R,=R - Ay: m, = m-h, w = 1 and 












Determine the smallest row 
index k such that rf =1, 
Is 2, +h. S ZO? 


Save this solution as the 
best discovered so far, 
Set’ Z° = 2° 

















Does there exist an index u, 
j <u< J., such that both 
RI@A,= A andzgtas 2° 2 
eE No 
Let v be the smallest such index u. 
Is 2S P= ho TF 







Figure 4, Flow chart of basic algorithm, 
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a list ij, i,» eels i, of the indicies of the w nonzero elements, bal : 
Otherwise each of the facets of the procedure are as has been described. 

As an illustration Table 1 gives the data for an example with m =5 
and n=31l. Upon determining the row indices a » the number of nonzero 
Stites h; and the average costs a= Cc; th; the vectors are sorted into 
sets and then ordered within sets, The resulting order is then as shown 
in the Table. Upon the completion of the ordering, the quantities 9; and 
J; for each row i are determined as is the quantity 6 j for each vector 
cr - The results are as shown in the Table. Search then commences, 

As can be verified by tracing through the procedure in Figure 4, 


the search process can be summarized by the tree in Figure 5 which shows 


the nonzero elements x; specified during the process. The values adjacent 





Figure 5. Tree of nonzero variables elaborated for illustrative 
problem of Table 1. 





= ehe 


Table 1, Data for Illustrative All- Zero-One Problem 
with m=5 and n=31 


OoMDAIKDUAWNH 


SCODCCCOORPHBHBHEPHEHEHH OD OOOKF OF OFRRFRFOHHE 
COOP FPREFRFRQODORPFHFKHOCOCOrPOrFOCOFRrFOOFrRHHOF 
OFFER ODO OOF COFRPHFHHFOFOCOrF OF SCOOFr CORR HEE 
FOF OFrFOOFCOrFOFrFOCOCOKrFCOrFROHRFOHFOFFE 
PENNWNREENNNWWAhWENNNWNWWWWWH AA AU 


UB BRWKWWWNNNNNNNNEPRP PEPE PEPE PP ee eee 


1 
1 
1 
1 
1 
1 
1 
1 
1 
1 
1 
1 
1 
1 
1 
1 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 


24 28 30 31 








SvElionss 


to the nodes are the costs incurred up to and including the specification 
of - =1; the asterisk indicates that R, = 0 and that a better feasible 
solution xo has been discovered, First x =1 is specified which 
results in the first feasible solution with Z° = 68, The process then 


backtracks to the origin and proceeds forward, specifying x, =1 followed 


2 
by Xorn =1. The result is a better feasible solution with Z° = 61, 
Backtracking to the previous node (x, =1), the test is made for a vector 
Uy 26< u < 28, such that R, ®@ Ay eAL .« Since no such vector 
is found the process backtracks again to the origin. The procedure then 
investigates in turn X3 = 1. X4 = tl. peaveress X19 = 1 finding in each case that 
R ® A, =A, and c, <61 but that c., (m,, = hy) « 6 ee 61, Finally, 
for v=11 the latter condition fails to hold so that xj, =1 is specified, 
followed by x2, =1. Determining the smallest index k such that 

r, =1 tobe 4, and 0, to be 28, the result is Ze + ,. >61. The pro- 
cedure therefore backtracks, setting Xo5 = 0, and proceeds to consider 
inturn x9¢ =1, Xoq = 1 and X53 =1 but in each case 40 + c,,+(3-h )b v> 6l. 
Hence, the procedure backtracks again to the origin and proceeds to consider 


in turn x =1,....% g= 1, In each case it is found that c, + (5-h))*6 >61, 


1 
At that point, since w=0, problem-solving is complete: the solution 
X,=1, X25 =1 is optimal. Of the 233 nodes ina complete tree of the 


problem, it was necessary for the procedure to explicitly evaluate but 25, 


TI. COMPUTATIONAL EXPERIENCE 

To investigatethe computational feasibility of the general approach 
computer programs were written in MAD (Michigan Algorithm Decoder) 
and a number of problems run on an IBM 7094 computer. For discussion 


purposes we denote as Algorithm 2 the procedure as flow charted in 





ea ee 


Figure 4. Algorithm 1 represents the same procedure with the two 
‘"knapsack" type tests employing the average costs 6 and number of 
remaining unsatisfied constraints m, omitted. : 

In Table 2 is shown the solution sienae” for a sample of thirteen 
problems arising in a truck dispatching context fa, 6, 17] . In'this 
context there is a central depot from which commodities are to be 
shipped by truck to m destinations. Vector a in (1) represents 


the j* 


truck route, the route starting at the depot and visiting each of 
the destinations i for which as = 1. All commodities for destination 
i are delivered in the same trip. The problem is to select a combination 
of routes with which to make all m deliveries at minimum cost, the 
cost of employing route j being ei, . 

Each of these sample problems were solved by both Algorithm 1 
and Algorithm 2, the solution times in Table 2 representing the time 
to discover an optimal solution and to prove its optimality. A comparison 
of times for the two algorithms indicates that the performance of the 
"knapsack" tests is a good investment of problem-solving time which 
frequently results in a major reduction in total solution time. For 


instance, in Example 10 the investment reduces total solution time to less 


than 1/18 of the problem-solving time required otherwise. 


6 The times shown are exclusive of computer input and output time and 
are accurate to + 1/60 of a second. 
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Table 2. Solution Times for Sample of Truck 
Dispatching Problems 


Problem Solution Times (Seconds) 


Size Algorithm Algorithm 


(m x n) 1 

5x31 . 050 

6x62 .167 

8x92 567 

13x91 35,167 6.367 
11x23] 9.183 1,383 
11x561 12.917 2,867 
11x1023 27,267 14, 383 
11x1485 34,950 19.317 
12x298 89,133 3,500 
12x538 131,033 7.117 
12x793 77, 667 4,567 


15x575 600. 000t 69. 483 


19x1159 2400, 0007 





+ Problem-solving terminated without having proved optimality 
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To obtain an indication of the efficiency of these algorithms re- 
lative to existing integer programming methods a number of problems 
were solved by Gomory's all-integer algorithm fi 1] on the IBM 7094 
using SHARE code IPM2 [zi . For those sample problems which were 
solvable with IPM2 (i. e., those for which n < 100) the solution time togeth-- 
‘er with the number of pivot steps required is shown in Table 2, From the 
limited computational experience gained with these algorithms to date the 
conclusion is that Algorithm 2 is frequently but not invariably preferred to 
the all-integer algorithm. For instance, in Example 1 Algorithm 2 required 
iene Bax V6 ofthetime of IPM2, but in Example 4 some 50%more, Fur- 
ther experience is needed to identify attributes of problems which would 
indicate a priori the preferred method to be used for problems of this size. 

As seen from the solution times in the Table, problem-solving 
time for the combinatorial programming algorithms tends to increase on 
the average both with the number of vectors n and the number of con- 
straints m inthe problem, For a specific problem, however, the time 
is quite unpredictable. For instance, problem-solving time required for 
Example 10 exceeds substantially that required for the larger problem of 
Example ll, where in fact the vectors in the smaller problem are a subset 
of those in the larger. 

As has been mentioned earlier, an important attribute of the com- 
binatorial programming algorithms is that search is directed first to the 
discovery of a feasible solution and then to successively better and better 
feasible solutions until ultimately one is discovered that is shown to be 
optimal, Hence it may be possible to terminate problem-solving with a 
usable (feasible) solution prior to the ultimate completion of the problem- 


solving process, 


7 This is the problem illustrated in the previous section, 


=. 20 = 


The importance of this attribute can be seen in Example 13 for which the 
process had not been completed after 40 minutes of computation on the 
IBM 7094, However, as shown in Table 3 problem-solving could have 
been terminated any time after the first 5.6 seconds with a feasible 
solution, In Table 3 is shown the rate of problem-solving progress for 
this example together with two others, For each example the Table shows 
the value of the objective function 7 for each successive feasible solution 
discovered together with the elapsed time t at which it was discovered.® 
The first elapsed time for each example includes both the time necessary 
to arrange the vectors from an arbitrary input order into the prescribed 
order for search and the subsequent time required to find a first feasible 
solution. As these results indicate the time required to find the first 
feasible solution tends in general to constitute a rather small fraction of 
the total problem-solving time. In addition, they sugge st? that a large 
fraction of the total time is commonly expended in proving optimality, a 
characteristic that has also been observed with other combinatorial programs,!9 
While most of the problems investigated to date have been of the truck 
dispatching type, a number of random problems have been solved with the 
algorithms, The resulting times for 6 problems with n=60 and 100, and 
m=1l0, 20 and 35 are shown in Table 4, In each case the problem was 
formed by generating (n-m) vectors A; with hy randomly distributed 
nonzero elements, by being uniformly distributed over the range 1 <hj<m, 
aa affixing to Trews (n-m) vectors the m units vectors, For all n 
vectors the cost c; was a uniformly distributed random variable in the 


range O0< es < 100. 


8 The difference in total time between Tables 1 and 2 for Examples 4 and 10 
is the time required to print intermediate solutions in the latter case. 
9 In Example 13 it is conjectured that Z = 683 is an optimal value. 


10 See, for instance, the experience reported in fs] with a class of 
sequencing problems, 
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Table 4. Solution Times for Sample of Randomly 
Generated Problems 


Solution Times (Seconds) 
Algorithm Algorithm 
2 


10x60 
10x100 
20x60 


20x100 


35x60 


35x100 





In contrast to the results for the truck dispatching type problems shown 
in Table 2, these results indicate that investment of problem-solving time in 
performing "knapsack"! tests is a poor investment: the added time spent in 
implementing the tests is not offset by a commensurate reduction in search 
time. Of the two algorithms, Algorithm lis thus preferred for the randomly 
generated problems, This difference in the effectiveness of the algorithms 
when applied to the two type of problems stems from differences in the charac- 
teristics of the problems, The truck dispatching problems are characterized 
by the facts that there tend to be a large number of feasible solutions to the 
problem and a high correlation between the hy and the es with the consequence 
that the lower bounds used in the tests tend to be close to the minimum attain- 
able cost for a feasible solution; hence the discrimination power of the tests 
tends to be sufficiently strong to effectively reduce search. On the other hand, 
for randomly generated problems the lower bounds tend to markedly under- 
state the attainable cost for feasible solutions, and hence to be less effective 
in pruning the tree. In the extreme case there is no resultant pruning, and 
total problem-solving time is simply extended by the time expended on fruitless 


testing. Thus for randomly generated problems this time should be eliminated, 
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or perhaps more effectively expended in performing additional feasibility 
tests, some of which are suggested in the following section, 

IV. MODIFICATIONS OF THE BASIC ALGORITHM 

While the computational feasibility of the general approach for solving 
all-zero-one type problems has been demonstrated with these basic algorithms, 
there are a number of simple modifications to these procedures which might 
result in more efficient procedures. 

For instance, Balinski [3] has noted that there are a number of pre- 
analysis considerations that might profitably be applied to the problem prior 
to the actual execution of the algorithms, First, in the absence of knowledge 
regarding the structure of the vectors sy comprising the problem, it would 
most likely be worthwhile to ascertain that for each row i, a; =1 for at least 
one j, for otherwise there exists no feasible solution to the problem, Secondly, 
it might be possible to reduce the number of vectors n in the problem through 
dominance considerations: vector A, may be eliminated from the problem if 


there exists vectors A. such that A, =A. and c. < c,. Thirdly, 
=ij Lids, =k Xi jj — 7k 


should there exist a row i with only a single vector A, having a nonzero ele- 
ment ass this row and column vector may be removed, yielding a smaller 
problem of dimensions (m-1) x (n-1) with requirements R! = R-A,. . Finally, 
if by permuting rows and columns of the matrix T formed by the column 
vectors a it is possible to form a matrix T' having the diagonal structure as 
shown in Figure 6, then an optimal solution can be obtained by decomposing 
the problem into the p subproblems defined by the diagonal submatrices T 
and solving each of them independently, 

Within the algorithms themselves, more efficient procedures may re- 


sult through use of additional feasibility andér dominance considerations, 
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Figure 6. Structural form of decomposable problem. All 
elements in the unshaded portion of the matrix are zero, 


For instance, it may be advantageous at each stage s to verify that the 
feasibility test 1 < m, is satisfied, where m , is the number of con- 


s<u< 
row index t for which rp = 1; whenever the test is not satisfied the search 


straints to be satisfied at stage s and Y = min fa for the smallest 

t 
process can backtrack immediately. Another feasibility consideration of 
potential value within the algorithm itself, especially when solving random 
problems, is that described earlier as a pre-analysis consideration wherein 
there must be at least one nonzero element ay for each i. Similarly, at 
each stage s during problem-solving there must exist for each i such 
that 2, = 1 at least one vector e » §< j <n, such that ai =l, This 
test can be easily implemented by defining recursively, after ordering the 
needs at the outset of the algorithm, the vectors S; » jen, n-l,...51, 


where E, = E, 
=) —jt+l 


ing, verifying at each stage s that R® E. =R,- 


@ 4; and En = A, ; and then, throughout problem-solv- 


With regard to dominance considerations, as was suggested in Section 
Il atest more stringent than 55+m_.< z, can be devised by using additional 
segments in the function 8 (u) shown in Figure 2 to get a higher lower 

A - : 


A 
bound Z. on the remaining cost Z . An alternative to this which would 





Y the symbol ® denotes the logical "or'' operation; as applied to vectors 
A= (aj, a71-+-, ay) and B = (by, b2,..+,by)) we mean by A @ B = C that 
if ah = 

a Oi aj b, 0 


for j = 4) 2) 0005s 
J ]1 otherwise 


= 2O= 


require considerably more setup effort, storage, and implementation time 
x, 
but which would be expected to yield higher lower bounds on Z, is to 


A i= q 
employ the bound Zs = ee i B » where - a0! fOr: 1) Sty 


Fett it 
A? , tsl,2,...,n-1 
min 14 ee ‘ yh | otherwise 


om fora, = 0 
in 


c otherwise 
n/h, 


This test would subsume the last feasibility consideration given in the pre- 
ceeding paragraph. Or, possibly it would prove more efficient to store and 
base tests only on the values A, for t=1, I, +1, 0 


tlre J 1, 


1 2 m-1* 

With the exception of the first and third pre-analysis considerations, 
the problem-solving time that would be consumed in implementing these 
modifications would not appear to be insignificant. Further investigation is 
required to determine whether, on the average, the reduction in search time 
occasionned by the use of these added considerations exceeds the time spent 
in implementing them, 

Finally, there are two general possibilities for improvement which 
are applicable to all combinatorial programming procedures employing a 
strategy of the type used in the present algorithms: pre- specification of 


upper and/or lower bounds on the value of the objective function, 2s ma”, 
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and searching by "successive approximations", Clearly if values of ZY 
and/or ae can be pre-specified to the algorithm at the outset,the possibility 
exists of reducing the amount of search time required by the algorithm - 
perhaps quite dramatically, as in cases when the values are close to or 
equal to the optimum value. In some instances such values may be readily 
supplied from previous experience, related problems, etc, without the necessity 
of expending significant amounts of problem-solving effort. More generally, 
this possibility for overall improvement through pre- specification of bounds 
leads to the general quest for "good" ways to get "good" initial solutions and 
bounds, and to the question of the optimal allocation of total problem-solving 
time between the preliminary search for "good" bounds to pre-specify to the 
algorithm and the execution of the algorithm per se. In the following section 
a strategy in which bounds are determined by linear programming is des-~- 
cribed, but in general the topic is beyond the scope of the present paper. 

In searching by "successive approximations" the basic algorithm is 
modified so that in searching for successively better and better feasible 
solutions it is now required that each feasible solution exhibit an improve- 
ment over its predecessor of at least an amount!“ x, By imposing this 
requirement it is hoped that greater portions of the tree will be pruned and 
problem-solving time thereby reduced, Upon termination a solution will 
have been discovered with a solution value Z* that is within’ of the 
optimum: ZS Z>Z*-~ SZ . At this point the algorithm can be re-applied, 


13 


if desired, to search the intermediate range for an optimal solution, or 


12 In the basic algorithm this modification consists simply in substituting 
the quantity (Z°-S7) for Z° in (3). 
= This strategy thus entails the re-elaboration and re-evaluation (at least 


implicitly) of portions of the tree that have already been considered earlier. 
With the enumeration procedures employed in the algorithms in this paper 
the reconsideration applies to paths in the tree corresponding to potential 


solutions lexicographically smaller than the solution which yielded the 
value Z". 
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for one within, gay of an optimum. 
Both of these general possibilities for improvement remain topics 
for future investigation. 


Vv. EXTENSION TO THE ALL-ZERO-ONE PROBLEM WITH INEQUALITIES 


With minor changes the discussion and algorithms for the all-zero-one 
problem of form (1) can be extended to the problem with inequalities of the 
' form given by (1'). In this last section we comment briefly on this extension, 
Paralleling the discussion in Section Il, we will assume in thé basic 
enumeration process for problems of form (2) that the column vectors A, 
are pre-ordered in a fixed order, and that potential solutions X=(x), Xgreees x,,) 
are generated in lexicographically decreasing order. Corresponding to (2) 
we have that at stage s-l in the enumerating process, having specified values 


Xp Ko reee at )» there remains a problem of the form (2'): 


Minimize’ ae — 
j 

Subject to: 1a 3 x, A> R- 3 LS a eR (2!) 
== I —s 

and all jes, stl,...,n 


From the nonnegativity of the elements a andthe inequality in (2) 
y 
it follows that for each k, 1 > n> ne aed rh > save >r > “ee - The 
xt inequaity becomes satisfied as soon as a stage t is reached for which 


Th < 0; thereafter in the formulation of problem (2') at stage s, s >t itis 


equivalent to set ty =0. That is, in general we set te = max [o, T. - 


Dstt Fj %] . »The importance of this equivalence is that we can thus 
continue to represent the elements of vectors R as bits in binary words. 


Computationally we now have for Ry 


R= Re y7 Be Fenn Ae. pe Beep QF ee) Dee 


=a 28 = 


where A 1 denotes the column vector whose elements are the binary 


complements of the elements of Ag-1 ‘ 


With this equivalence the basic ennumeration procedure for pro- 


blems of form (2) becomes: 
(i) Set *%) = F (Ry); *2 =F (Ry)s...5 x, = F (R,). 


If RL = 0 save X as the best feasibly solution a 
c, x 


at"j - Go to (ii). 


discovered so far, setting Zo Mh 

(ii) If there isa j, 1< j <n such that *, =1 let s be the 
largest such j and go to (iii), Otherwise problem-solving 
is complete, 


giveed Xo. = Xe_45 ¥, = 03 X= F (Ro); 


(iii) Redefine at =X} X= x 
ae CG, Hae Z° save X 
a 5 j = 


as the best solution discovered so far. Go to (ii). 


st A SB (R..), If R, = and oy 


14 
where 0 is the m-dimensional column vector with all elements zero and 


0 if R A_= 0 
F (R,) = =8 =5 a 


1 otherwise 
Continuing on, it follows as in the case of problem (2) that a feasible 
solution can exist to problem (2') only if there exists for each k such that 


rel at least one vector es with a,, =1, t>s., This consideration can be 


kt 


simply implemented in a manner similar to that used for problem (2) by 
/ 
ies k 


ordering vectors on the basis of these sets. In the present case, however, 


appropriately defining sets S. Shag teacgt of the vectors A, and pre- 
pprop y 2 ma 4 P 


this is accomplished at the expense of a marked increase in problem sive? 
resulting from the replication of vectors A, For the present problem 


14 That *,=0 when R A. =0 follows from dominance considerations, and 
that X, =] when R.@A, 70 from feasibility considerations, 

15 Alternately it can be accomplished at the expense of additonal bookkeeping 
which is probably more efficient computationally, especially when m > B, 


The present discussion is facilitated, however, by assuming the vectors 
explicitly replicated, 
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sets st, St eet) st are defined such that S/’contains all vectors for which 


ar =l. If vector A, is then replicated (iP a) times and placed in each 


of the relevant sets, andthe N = » vectors pre-ordered so that those 


jk “ik 


in set S/ precede those in set Ss; etc., we can then proceed exactly as in 
1 P P y 


2 


case of problem (1). Thatis, if jt is the number of vectors in sf » and Be = 
dee Je » then the search process after specifying Xe s=J; » i=l,2,...,me-l, 


s 
can backtrack if re 1, Also, in the forward search process if t denotes 


stl 


the smallest index i such that bt =l at the completion of stage s, then 


explicit investigation can jump to stage s’=J, ,+l1 and resume with R,.aR 


t-1 stl’ 


Proceeding to dominance considerations we have as before: (i) 

x, 70 if cl > 3 (ii) search may backtrack at stage stl if ® > Zest? where 

X= min feJana t is the smallest index i such that rt =]; and 
J 


1st A 
(iii) search may backtrack at stage s if c. > zs or 2 > a where 2. is 


>» 


a lower bound on the value of an optimal solution Ze to the knapsack problem 
defined by (4) with the constraint ae ma ¥; =m, replaced by the inequality 
Dee hy be > m,. In this latter instance all of the bounding considerations 
ees awed in SectionII apply to the present problem, In the present case the 
quantity hy for vector baa is redefined to be h, = De as; where ay isa 
member of eet sf . 

Finally, turning to the modifications in SectionIV each of Balinski's 
four pre-analysis considerations apply when the second is appropriately 
changed for the present problem: vector A. may be removed from the pro- 
blem if there exists vectors S, such that i A > rm and yy c; < a 
Within the algorithm itself the test for . > 25 remains ee 255 
a, . (Again the quantity he appearing in the expressions for Bi 
is redefined, as above.) And, of course, both of the general possibilities 


for improvement apply to the present problem: pre~specification of upper 


or lower bounds on the objective function, and searching by ''successive 


approximations,"' 
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As an illustration we consider again the problem with m=5 and 
n=3lin Table 1, Applying Balinski's second pre-analysis dominance con- 
sideration twenty of the original vectors oF are eliminated, leaving vec- 
torsy = 2, 6, 7,10,11,13,17, 21, 22,25 and 31, Upon replicating each of 
these remaining vectors as to form the sets Si, Si fare sf there results 
bi ae Bia = 26 vectors as . Sorting these vectors into sets, and order- ¥ 
ing within sets in increasing value of a, = ohh, , there results the equiva- 
lent problem whose data is given in Table 5, er with the solution 
of the problem, the result is as indicated by the tree in Figure 7 which 
shows the non-zero variables specified during the process, First x,=1 


ak 


is specified, followed by x,, =1 to yield a feasible solution with Z=6l. 


14 
The process then backtracks to the node at x) 5 l and considers in turn 
j=15 and 16 but finds in bothcases z,7 cs > 61. Hence the process back- 
tracks to the origin and investigates inturn j=2,3 and 4 finding that ws 61 
but that S + (m - By) : a; >6l. For j=5 etanch,y: 8, < 61 so that we specify 
x, = 1 followed by X14 = l. Then, however, Ze +My 46 + 26> 61 so the 


process backtracks to the node x, =1 andtries j=15. Since eu e5 =63>61 


5 
it backtracks again to the origin. For j=6, et (m-h¢) 5 = 45 + 3(6) > 61 


and, since w=o, problem-solving is complete. 





61% 


46 


Figure 7. Tree of nonzero variables elaborated for 
illustrative problem of Table 5, 


= 3). 


Table 5, Data for Illustrative All- Zero-One Problem 
With Inequalities as Derived from Problem 
in Table 1 

















6. 00 
6.00 
6, 00 
6. 00 
6. 00 
6. 00 
6. 00 
6. 00 
6, 00 
6, 00 
6.00 
6.00 
6, 00 
6. 00 
11. 00 
11.00 
11, 00 
11, 00 
11. 00 
11, 00 
11.00 
32, 00 
45.00 
50. 00 
54,00 
55.00 
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In conclusion, we comment briefly on possibilities for improve- 
ment through pre-specification of bounds. As remarked in the previous 
section it may in practice be more efficient to first obtain by some means 
upper and lower bounds for the optimal value of Z, Zz and By and then 
employ the combinatorial programming procedure for reliably searching the 
intermediate range. For the present problem both bounds can be readily 
derived from an optimal noninteger solution to the problem given in (2) 


as determined by a linear programming algorithm. [If Le (Yessy en's 1Y,) 


see an optimal noninteger solution to (2) fie og = a 


and zu “ya iy where y, =<y, + .99. wD Ds the ee ey 
j 
denoting the hae “ integer ecetne in <? ° 
As an alternative, problem-solving time might first be spent in 


searching for a rounded feasible integer solution W = (ws Wesg evel wi) 


yg 
n is 
to (2) with value Lijai <; w, closer to optimum than Y = Gp,» siege Mi 
7 J 
L U , 
thereby reducing the range Z < Z<Z_ to be searched, With reference 


again to the optimal noninteger solution Y, the solution W= (w pWorees »W,) 


derived by setting Ww, at, .») for all j constitutes an optimal integer solu- 


tion to the problem with requirements a % 4; . With these require- 
ments fulfilled by W there remains a weastibs problem of the form of (2) 
with requirements R - ye F A; . if We (WW, ses Ww) represents 
any feasible. solution to this remaining problem then w=Wtw constitutes a 
feasible solution to the original problem having requirements R. if 

ge pS c.w < hi ea 6, We. 2 20s ee a Ww, solution W constitutes 


a feasible solution with value Z closer to optimal than Y; in the special 


event that De. cw gi Loa ae solution W constitutes an optimal 
JE 1 


solution, 


16 In the event that all coefficients cj are integers, the higher lower bound 
dacs en yy 199 > may be used, 


= 33 = 


Pursuing this latter alternative, a reasonable problem-solving 
strategy would appear to be to first obtain an optimal noninteger solu- 


tion Y for the original problem, Then from Y formulate the remain- 
jan — 
j=l 


. C. w, and ae pee c, w, , and search by the combinatorial pro- 
jal Jj jal joj S 


ing subproblem with requirements R- 


ae 


A; with bounds 2h a 
jen | 
gramming procedure the intermediate range, obtaining:an optimal 
integer subproblem solution W*! Forming 'W = W+¥* we test if 

a 3 my = ay if so, problem-solving is complete; silicate the 
combinatorial programming procedure is applied to the original pro-.. 
blem to investigate the range gn <Z< ae e my . In the context of 
the basic one-dimensional cutting stock problem C14] this.geneval 

_ strategy was found to be quite successful in that for each of the approxi- 
mately one hundred cases investigated, solution W constituted an 
optimal solution, making it unnecessary to subsequently apply the com- 
binatorial programming procedure to the original problem. For the 
present class of all-zero-one problems,possibly the investigations of 
Balinski and Norman (see [4] ) will yield information regarding the 


effectiveness of problem-solving strategies of this type. 
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